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professional  team,  recognized  as  leaders  in  our  field. 
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Results  in  Brief 

Exoatmosphehc  Kill  Vehicle  Quality  Assurance  and 
Reliability  Assessment  -  Part  A 


September  8,  2014 


Findings  (cant'd) 


Objective 

Our  objective  was  to  perform  a  quality 
assurance  assessment  of  the  Missile  Defense 
Agency's  Ground-Based  Midcourse  Defense, 
Exoatmospheric  Kill  Vehicle,  which  is 
procured  from  Raytheon  Missile  Systems  via 
the  prime  contractor  Boeing.  Our  assessment 
resulted  in  two  separate  reports. 

Part  A:  Assess  Raytheon  conformity  to 
Aerospace  Standard  (AS)9100C,  “Quality 
Management  Systems  -  Requirements  for 
Aviation,  Space  and  Defense  Organizations," 
contractual  quality  assurance  clauses,  and 
internal  quality  assurance  processes  and 
procedures. 


B.  Boeing  and  Raytheon  did  not  ensure  all  quality  assurance 
and  technical  requirements  for  mission-critical  assemblies 
flowed  down  to  the  supply  chain  and  were  verified. 
Therefore,  it  is  uncertain  that  all  supplier  products  will 
meet  system,  performance,  and  reliability  requirements. 

C.  Boeing  and  Raytheon  were  not  adhering  to  configuration 
management  processes,  specifically  with  respect  to 
management  of  change  processes  for  design  requirements. 
This  leads  to  some  uncertainty  in  fielded  configurations. 

D.  Missile  Defense  Agency,  Boeing,  and  Raytheon  were  not 
ensuring  that  all  quality  management  systems  were  in 
compliance  with  the  AS9100C  standard.  We  identified 
a  total  of  48  nonconformances  that  were  violations  of 
the  AS9100C  standard.  These  nonconformances  could 
result  in  the  production  of  nonconforming  hardware 
and  software  which  could  effect  mission  success. 


Part  B:  Assess  the  Exoatmpheric  Kill  Vehicle 
reliability  of  deployed  assets.  It  will  be 
released  as  a  separate  classified  report. 


Recommendations 

The  Director,  Missile  Defense  Agency,  should: 


Findings 

The  majority  of  quality  management  systems 
were  in  compliance.  However,  some  areas 
need  improvement: 

A.  Boeing  and  Raytheon  were  not 
ensuring  that  software  development 
processes  and  testing  were  sufficient, 
which  could  result  in  reliability  issues. 


A.  Ensure  software  development  processes  are  fully 
documented,  implemented,  and  enforced  throughout  the 
Exoatmospheric  Kill  Vehicle  supply  chain. 

B. l  Ensure  all  suppliers  of  critical  items  are  identified  as 

critical  suppliers,  receive  the  necessary  contractual 
requirements,  and  requirements  are  verified  throughout 
the  supply  chain. 

B.2  Ensure  fielded  hardware  affected  by  an  insufficient 
Hardware  Acceptance  Review  Checklist  process  is  assessed 
for  risk. 
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Recommendations  (cont'd) 

C.  Ensure  design  and  configuration  changes  do  not 
circumvent  the  Missile  Defense  Agency  Assurance 
Provisions  for  configuration  management. 

D.  Conduct  an  effective  root  cause  analysis 
and  implement  corrective  actions  for  all  48 
nonconformances  including  assessing  the  risk  to 
fielded  hardware. 


Management  Comments 

MDA  provided  responsive  comments  on  the  draft  of  this 
report  and  agreed  with  the  five  recommendations  in  the 
report.  Additionally  MDA  provided  technical  and  security 
comments  that  have  been  incorporated  into  this  report 
as  appropriate.  No  further  comments  are  required. 
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MEMORANDUM  FOR  DIRECTOR,  MISSILE  DEFENSE  AGENCY 

SUBJECT:  Exoatmospheric  Kill  Vehicle  Quality  Assurance  and  Reliability  Assessment  -  Part  A 
(Project  No.  DODIG-2014-111) 

The  DoD  Office  of  Inspector  General  (OIG)  conducted  a  quality  assurance  and  reliability  assessment 
of  the  Ground-Based  Midcourse  Defense  (GMD),  Exoatmospheric  Kill  Vehicle  (EKVJ,  procured  from 
Raytheon  Missile  Systems.  Our  assessment  resulted  in  two  reports,  Part  A  and  Part  B.  Part  A, 
assessed  the  GMD  EKV  program’s  quality  management  system.  Part  B  is  classified  and  provides 
our  reliability  assessment  of  deployed  GMD  EKV  assets. 

We  found  for  the  GMD  EKV  that  the  majority  of  quality  management  systems  were  in 
compliance.  However,  the  Missile  Defense  Agency  (MDA)  was  not  ensuring  that  all  quality 
management  systems  provisions  complied  with  the  AS9100C  standard  which  could 
inadvertently  effect  mission  success.  We  identified  a  total  of  48  nonconformances  that  violated 
the  AS9100C  standard.  The  nonconformances  indicated  that  MDA  needs  to  improve  EKV 
quality  assurance  processes  in  the  following  areas:  software  development  and  testing,  flow 
down  of  quality  assurance  and  technical  requirements  for  mission-critical  assemblies,  and 
management  of  the  engineering  change  process  for  design  requirements. 

We  considered  management  comments  to  the  draft  report  when  preparing  the  final  report. 
The  comments  received  from  MDA  conformed  to  the  requirements  of  DoD  Directive  7650.3; 
therefore,  we  do  not  require  additional  comments. 

We  appreciate  the  courtesies  extended  to  our  staff.  Please  direct  questions  to  Mr.  Thomas  Bulk 
at  (703J  604-9619  or  thomas.buIk@dodig.mil.  If  you  desire,  we  will  provide  a  formal  briefing  on 
the  results. 


Randolph  R.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 
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Objectives 

Our  objective  was  to  perform  a  quality  assurance  assessment  of  the  Missile  Defense 
Agency’s  Ground-Based  Midcourse  Defense  (GMD),  Exoatmospheric  Kill  Vehicle  (EKV], 
which  is  procured  from  Raytheon  Missile  Systems.  We  conducted  the  assessment  onsite 
at  the  GMD  Program  Office,  Huntsville,  Alabama,  and  at  Raytheon  Missile  Systems, 
Tucson,  Arizona.  Based  on  our  evaluation  of  program  office  data,  we  determined 
it  necessary  to  review  EKV  reliability  in  conjunction  with  our  quality  assurance 
assessment.  Thus,  our  assessment  resulted  in  two  parts: 

•  Part  A  (Unclassified]:  Assess  Raytheon  conformity  to  Aerospace  Standard 
(AS]9100C,  "Quality  Management  Systems  -  Requirements  for  Aviation, 
Space  and  Defense  Organizations,"  contractual  quality  assurance  clauses, 
and  internal  quality  assurance  processes  and  procedures. 

•  Part  B  (Classified):  Assess  the  reliability  of  deployed  assets  by  evaluating 
historical  manufacturing  and  quality  management  system  data. 

This  report  is  Part  A  of  our  assessment;  Part  B  will  be  classified  and  released  at  a 
later  date. 

Background 

Establishment  of  Missile  Defense  Agency 

A  Secretary  of  Defense  memorandum  titled,  "Missile  Defense  Program  Direction," 
January  2,  2002,  established  a  single  program  to  develop  an  integrated  defense 
capability  under  the  management  of  Missile  Defense  Agency  (MDA).  MDA  is  responsible 
for  developing  the  various  missile  defense  programs  identified  to  support  an  overall 
Ballistic  Missile  Defense  System  (BMDS)  (Figure  1). 

The  GMD  is  part  of  the  BMDS  and  is  an  integral  part  of  the  layered  defense  strategy. 
According  to  MDA  the  GMD’s  mission  is  to  "provide  Combatant  Commanders  a  capability 
to  engage  and  destroy  limited  intermediate  and  long-range  ballistic  missile  threats  in 
the  midcourse  battle  space  to  protect  the  U.  S.  Homeland." 
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Figure  1.  Ballistic  Missile  Defense  System 
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Acquisition  Strategy 

The  Secretary  of  Defense  memorandum  of  January  2,  2002,  states  that  the  BMDS  is 
not  subject  to  the  traditional  requirements  generation  process  of  the  Joint  Capabilities 
Integration  and  Development  System.  Therefore,  MDA  was  authorized  to  use 
non-standard  approaches  for  both  acquisition  and  requirements  generation.  MDA’s 
acquisition  processes,  however,  were  and  are  required  to  be  consistent  with  the 
principles  of  DoD  Directive  (DoDD)  5000.01,  "The  Defense  Acquisition  System,”  and 
DoD  Instruction  (DoDIJ  5000.02,  "Operation  of  the  Defense  Acquisition  System".  When 
the  interceptor  was  acquired,  MDA  did  not  have  a  standardized  acquisition  process. 
On  January  13,  2009,  MDA  established  its  acquisition  policy  to  standardize  MDA’s 
acquisition  processes.  On  April  29,  2011  MDA  updated  its  acquisition  management 
policy  to  reflect  standard  acquisition  lifecycle  phases.  MDA  has  subsequently 
updated  its  acquisition  policy  in  MDA  Directive  5013.01  (August  24,  2012]  and  MDA 
Instruction  5013.02-INS  (August  24,  2013]. 
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Because  of  the  urgent  need  and  Presidential  directive  to  field  a  capability,  the 
GMD  Program  expedited  the  EKV  acquisition  process.  DoDD  5134.09  requires 
MDA  to  manage  the  BMDS  consistent  with  the  principles  of  DoDI  5000.02. 
However,  DoDD  5134.09  allows  an  accelerated  acquisition  and  flexible  approach  to 
requirements  generation  and  acquisition  processes  instead  of  the  standard  processes 
identified  in  DoDI  5000.02.  Therefore,  the  EKV  did  not  go  through  the  milestone 
decision  review  process  and  the  product  development  phase  (Engineering  and 
Manufacturing  Development).  The  purpose  of  the  milestone  decision  review  is 
to  carefully  assess  a  program’s  readiness  to  proceed  to  the  next  acquisition  phase 
and  to  make  a  sound  investment  decision  committing  the  DoD’s  financial  resources. 
For  the  product  development  phase,  the  program  is  assessed  to  ensure  that  the 
product  design  is  stable,  manufacturing  processes  are  controlled,  and  the  product  can 
perform  in  the  intended  operational  environment. 

GMD  Program  data  indicated  that  the  EKV  prototype  design  was  forced  into  operational 
capability  with  minimal  design  turns,  and  only  those  addressing  obsolescence 
were  undertaken  by  the  Program.  The  design  turns  primarily  addressed  the  most 
critical  obsolescence  performance  risks  versus  addressing  reliability,  producibility, 
and  maintainability  for  the  operational  fleet. 

Ground-Based  Midcourse  Defense  System 

The  GMD  Program,  which  is  part  of  the  Ballistic  Missile  Defense  System,  was  initiated 
in  the  1990s  to  develop  a  homeland  missile  defense  system  against  rogue  nations. 
Using  space,  ground,  and  shipboard  sensors,  the  GMD  battle  management  system 
assesses  the  threat,  determines  if  the  threat  exists  in  its  battle  management  space, 
and  launches  an  interceptor  to  intercept  and  destroy  the  warhead  in  flight.  Today,  the 
GMD  system  is  composed  of  30  Ground-Based  Interceptors  (GBIs)  located  in  missile 
fields  in  Fort  Greely,  Alaska,  and  Vandenberg  Air  Force  Base,  California,  with  fire  control 
nodes  in  Colorado  and  Alaska.  A  memorandum  released  in  March  of  2013  by  the 
Secretary  of  Defense  sought  to  increase  the  number  of  GBIs  by  14  for  a  total  of  44. 
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Figure  2  shows  a  three-stage  GBI  with  the  EKV  as  the  payload.  The  booster  portion 
of  the  GBI  carries  the  EKV  toward  the  target’s  predicted  location  in  space.  Once 
released  from  the  booster,  the  EKV  uses  guidance  data  transmitted  from  Ground 
Support  and  Fire  Control  System  components  and  onboard  sensors  to  identify  and 
destroy  the  target  warhead.  EKV  impact  is  outside  the  Earth’s  atmosphere  using  the 
direct  collision  to  destroy  the  target  warhead.  Boeing  is  the  prime  contractor  for 
the  GBI  and  procures  the  EKV  from  Raytheon  Missile  Systems.  There  are  primarily 
two  variants  of  the  EKV  system,  capability  enhancement  I  (CE-I)  and  capability 
enhancement  II  (CE-II).  The  CE-I  configuration  includes  connector  upgrades  from 
the  first  prototype  to  address  obsolescence  issues.  The  CE-II  version  is  an  upgrade  of 
CE-I  to  resolve  processor  obsolescence  issues  and  to  enable  the  EKV  to  track  a  greater 
number  of  objects.  There  are  other  subconfigurations  within  CE-I  and  CE-II  variants 
that  resulted  from  resolving  design  or  manufacturing  risks. 


Figure  2.  Ground-Based  Interceptor 
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Source:  MDA  GMD  Program  Overview,  November  4,  2013 


GMD  Program  Schedule  Impacts 

Expedited  Delivery  Schedule 

National  Security  Presidential  Directive-23  [NSPD-23],  "National  Policy  on  Ballistic 
Missile  Defense,"  December  16,  2002,  directed  the  Department  of  Defense  to  deploy  a 
set  of  initial  missile  defense  capabilities  beginning  in  2004.  NSPD-23  resulted  in  the 
fielding  of  initial  missile  defense  capabilities  before  rigorous  testing  was  complete  to 
validate  performance.  Schedule  constraints  also  necessitated  the  need  to  field  GMD 
prototype  assets. 

The  current  EKV  design  is  the  prototype  design  of  1998  with  upgrades  for  design 
and  manufacturing  defects,  and  obsolescence  issues.  The  immediate  need  for  an  initial 
capability  drove  an  accelerated  development  process  and  fielded  capability  before 
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EKV  performance  was  fully  characterized  prior  to  initial  fielding.  Requirements  were 
viewed  as  "goals"  with  little  focus  on  reliability,  producibility,  and  maintainability 
requirements,  which  are  integral  to  strategic  systems  with  a  life  expectancy  similar 
to  GMD. 

A  combination  of  cost  constraints  and  failure-driven  program  restructures  has  kept 
the  program  in  a  state  of  change.  Schedule  and  cost  priorities  drove  a  culture  of 
"Use-As-Is”  leaving  the  EKV  as  a  manufacturing  challenge.  With  more  than  1,800  unique 
parts,  10,000  pages  of  work  instructions,  and  130,000  process  steps  for  the  current 
configuration,  EKV  repairs  and  refurbishments  are  considered  by  the  Program  to  be 
costly  and  problematic  and  make  the  EKV  susceptible  to  quality  assurance  failures. 

Flight  Tests 

Ten  flight  tests  were  completed  since  2005  with  seven  tests  designed  to  be  intercept 
tests  and  three  as  non-intercept  tests.  Of  the  seven  intercept  tests  flown,  four  were 
CE-I  and  three  were  CE-II  configurations.  Three  of  these  intercept  tests  resulted  in 
failures  attributable  to  the  EKV.  The  lessons  learned  from  the  failed  tests  led  to  a 
series  of  hardware  and  software  design  changes.  Overall,  these  issues  resulted  in 
the  GMD  Program  suspending  and  slowing  production  of  interceptors. 

Flight  Test  Failures 

Until  the  recent  successful  FTG-06b  intercept  flight  test  completed  on  June  22,  2014, 
the  GMD  Program  had  been  unable  to  demonstrate  a  successful  CE-II  Interceptor 
test.  Flight  Test  GMD  06  (FTG-06)  in  January  2010  failed  due  to  a  missing  lockwire 
on  a  CE-II  EKV  wire  harness  connector.  Work  instructions  were  updated  as  part  of 
the  corrective  action  to  ensure  verification  of  lockwire  connectors.  GMD  added  a 
retest  designated  as  FTG-06a.  However,  the  retest  also  failed  in  December  2010 
due  to  the  effects  of  vibration  on  the  EKV  guidance  system.  Due  to  the  flight  test 
failures,  MDA  halted  GMD  flight  testing,  restructured  its  flight  test  program,  stopped 
production  of  the  GMD  interceptors,  and  redirected  resources  to  return-to-flight 
testing  activities.  A  CE-I  flight  test  was  conducted  in  July  2013  (FTG-07)  to  support 
the  return-to-flight  testing  activities,  which  resulted  in  a  test  failure.  To  date,  the 
FTG-07  failure  investigation  is  still  ongoing  to  determine  the  root  cause.  The  scope 
of  the  investigation  encompasses  the  entire  EKV  and  Orbital  Boost  Vehicle  power 
system.  Table  1  and  Figure  3  provide  a  summary  of  EKV  flight  test  events  and 
failures  including  those  noted  in  FTG-06,  FTG-06a,  and  FTG-07. 
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Table  1.  EKV  Flight  Test  Events  and  Failures  for  CE-1  and  CE-II 


Mission 

Type  Test 

Success 

EKV  Flight 
Issue 

EKV  Corrective 
Action 

Planned 

Intercept 

Version 

Date 

FT-1 

No 

Target 

Yes 

Inertial 

Measurennent 
Unit  and 

Sensor  Off 

Nonninal 

Performance 

Software 

Improvements 

No 

CE-I 

13  Dec  05 

FTG-02 

Intercept 

Yes 

Inertial 
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Figure  3.  EKV Flight  Test  Events  and  Failures  CE-1  and  CE-11 
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Contract 

The  Boeing  Company  was  awarded  the  prime  contract  (HQ0006-01-C-0001) 
on  January  1,  2001,  for  evolutionary  development,  integration,  and  testing  of  a 
cost-effective  GMD  system.  Boeing  later  awarded  Raytheon  Missile  Systems  a 
subcontract  for  the  EKV.  Boeing  required  Ra3d:heon  to  comply  with  AS9100  Quality 
Standards  in  accordance  with  statement  of  work  D743-11961-2  dated  June  7,  2011. 
There  were  originally  33  Test  Bed,  CE-I  EKVs  that  were  delivered  from  2004  to 
October  of  2007  and  24  CE-II  EKVs  planned  to  be  delivered  between  2008  and 
2015,  of  which  16  have  been  delivered  to  date.  On  December  30,  2011,  Boeing  was 
awarded  a  Development  and  Sustainment  Contract  (DSC],  HQ0147-12-C-0004,  to 
develop  new  capabilities,  manufacture,  test,  and  provide  operational  support  of  the 
GMD  system.  Boeing  then  awarded  Ra3d:heon  the  subcontract  for  the  EKV  in  2012. 
The  contract  scope  includes  resolving  reliability  and  obsolescence  issues;  extending 
the  EKV  service  life;  improving  fleet  reliability  with  new  CE-II  Block  1  EKVs  and  CE-I 
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upgrades;  increasing  software  and  modeling  and  simulation  capabilities;  and  providing 
EKV  operations  and  flight  test  support  through  2018.  Boeing  and  mission-critical 
suppliers,  including  Raytheon,  were  required  to  meet  AS9100  and  the  MDA  Mission 
Assurance  Provisions  [MAP). 

The  MAP  provides  a  measureable  standardized  set  of  Quality,  Safety,  and  Mission 
Assurance  requirements  to  be  applied  to  those  suppliers  developing  mission  and 
safety  critical  items.  Under  the  Development  and  Sustainment  Contract,  Raytheon 
is  required  to  provide  a  MAP  Requirement  Applicability  Matrix  [RAM]  that  reflects 
MAP  implementation  to  ensure  planning  and  execution  of  the  Development  and 
Sustainment  Contract  [HQ0147-12-C-0004)  is  consistent  with  the  MDA  Assurance 
Provisions.  The  RAM  specifies  applicability  and  approaches  to  MAP  requirements. 
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Quality  Assurance  Assessment 

AS9100C  Quality  Management  Systems  -  Requirements 
for  Aviation,  Space,  and  Defense  Organizations 

We  evaluated  the  Exoatmospheric  Kill  Vehicle  quality  assurance  processes  at  Raytheon 
Missile  Systems,  Tucson,  Arizona,  to  the  AS9100C  Quality  Management  System 
standard,  as  implemented  by  MAP  Revision  A.  The  majority  of  systems  were  in 
compliance  with  AS9100C.  However,  some  areas  needed  improvement.  We  identified 
a  total  of  48  nonconformances  as  violations  of  the  AS9100C  standard.  Based  on  these 
nonconformances,  we  determined  that  MDA,  Boeing,  and  Raytheon  were  not  ensuring 
that  all  the  quality  management  systems  were  in  compliance  with  the  AS9100C 
standard.  Based  on  AS9101D  definition,  we  classified  each  of  the  nonconformances  as 
major  nonconformances  or  minor  nonconformances.  Each  nonconformance  received 
an  additional  technical  review  for  accuracy  and  classification.  This  report  focuses  on 
the  major  nonconformances. 

Although  we  identified  a  total  of  48  nonconformances,  some  systems  were  compliant 
as  stated  above.  Facility  shop  floor  operations  that  included  processes  such  as  tool 
control,  foreign  object  debris  control,  and  electrostatic  discharge  prevention  were  in 
place  and  controlled.  The  electronic  manufacturing  process  documentation  system 
appeared  to  ensure  that  out-of-sequence  work  and  manufacturing  liens  (controls  on 
incomplete  or  nonconforming  material)  were  addressed.  Integration  of  the  EKV  was 
done  using  clean  room  operations,  which  is  typical  of  high  reliability  space  programs. 

In  accordance  with  AS9101D  (Quality  Management  Systems  Audit  Requirements  for 
Aviation,  Space,  and  Defense  Organizations)  standard,  a  major  nonconformance  is 
a  nonfulfillment  of  a  requirement  that  is  likely  to  result  in  the  failure  of  the  quality 
management  system  or  reduce  its  ability  to  ensure  controlled  processes  or  compliant 
products/services.  A  minor  nonconformance  is  a  nonfulfillment  of  a  requirement  that 
is  not  likely  to  result  in  the  failure  of  the  quality  management  system  or  reduce  its 
ability  to  ensure  controlled  processes  or  compliant  products  or  services. 
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Table  2  shows  the  breakdown  of  major  and  minor  nonconformances  from  the 
assessment. 


Table  2.  Major/Minor  Nonconformances  for  Each  Organization 


Organization 

Major 

Minor 

Raytheon 

15 

25 

Boeing 

6 

1 

MDA  GMD  Progrann  Office 

1 

0 

Shown  in  Figure  4  are  the  nonconformances  by  AS9100C  clause  and  affected 
organization. 


Figure  4.  Nonconformances  by  AS9100C  Clause 
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The  following  sections  discuss  major  nonconformances  or  an  aggregate  of  minor 
nonconformances  considered  collectively  to  constitute  greater  risk. 

Raytheon 

Design  and  Development  (7.3) 

Design  and  Development  Planning  (7.3.1).  Raytheon  was  not  compliant  with  the 
required  embedded  coding  standard.  Programmers  used  lower  case  letters  rather 
than  the  required  uppercase  letters  when  declaring  alphanumeric  hexadecimal 
numbers.  Upper  case  letters  are  easier  to  differentiate  from  numbers  and  vice  versa, 
for  example,  the  letter  "1"  could  be  confused  with  the  number  "1.”  Not  following  the 
coding  standard  poses  a  performance  risk  to  future  code  modifications  or  maintenance. 
Another  nonconformance  noted  that  the  number  of  executable  statements  exceeded 
the  maximum  of  600  statements  as  allowed  by  the  EKV  Coding  Standard  for  code 
files.  This  included  the  health  and  status  test  code  file  of  the  kill  vehicle  mission-critical 
hardware,  in-flight  control  software,  the  interrupt  management  and  processing  file, 
and  the  kill  vehicle  communication  processing  file.  File  sizes  that  exceed  established 
standards  could  impact  execution  times  and  result  in  degraded  performance 
and  maintenance. 

Although  Raytheon  had  information  assurance  processes  in  place  to  comply  with 
DD  Form  254  and  DoD  5220. 22-M,  Raytheon  did  not  have  processes  in  place  that 
comply  with  DoD  Information  Assurance  Certification  and  Accreditation  Process  as 
required  by  DoDI  8500.2,  "Information  Assurance.”  This  increases  EKV  information 
risk  of  unauthorized  access,  information  inaccuracy  and  inconsistency,  and  data 
unavailability.  Another  nonconformance  identified  that  some  of  the  required 
safety  critical  software  test  cases  were  not  conducted  for  CE-I  Embedded  Software 
Build  20.8,  Delta  Formal  Qualification  Testing.  Only  10  out  of  30  of  the  required 
test  cases  were  conducted.  Formal  qualification  testing  is  performed  to  fully  verify 
software  performance  under  all  possible  conditions  or  scenarios.  Failure  to  perform 
full  qualification  testing  can  lead  to  loss  or  significant  degradation  of  the  mission. 

Design  and  Development  Inputs  (7.3.2).  Raytheon’s  Failure,  Modes,  Effects,  and 
Criticality  Analysis  (FMECA)  report  did  not  reflect  the  current  EKV  design.  The 
critical  items  list  contained  legacy  part  numbers,  and  had  not  been  updated  to 
reflect  the  current  design.  Several  of  those  critical  items  were  no  longer  valid  due  to 
design  changes  with  the  Inertial  Measurement  Unit.  FMECA  is  an  analytical  method 
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that  is  instrumental  in  the  design  process  to  identify  failure  modes  and  mitigate 
risks.  An  inaccurate  FMECA  can  result  in  not  identifying  failure  modes  that  should 
be  mitigated.  The  FMECA  should  be  updated  and  or  assessed  for  each  configuration 
change  to  prevent  unintended  consequences. 

Design  and  Development  Verification  (7.3.5).  There  was  no  evidence  that  some 
software  unit  test  results  were  evaluated  for  requirements  traceability,  internal  and 
external  consistency  of  requirements,  test  coverage,  compliance  to  work  instructions, 
feasibility  of  software  integration  and  testing,  and  feasibility  of  operation  and 
maintenance.  Evaluating  these  items  is  essential  to  ensuring  there  are  no  quality 
escapes  that  will  degrade  functionality,  performance,  and  maintenance  of  the  software. 

Control  of  Design  and  Development  Changes  (7.3.7).  Ra3d:heon  inappropriately 
used  the  waiver  and  deviation  process  to  change  the  production  baselines  for  some 
CE-1  and  CE-11  subcomponents.  Some  of  the  waivers  and  deviations  were  incorrectly 
classified  as  minor  rather  than  major;  therefore,  additional  engineering  analysis 
by  Boeing  or  MDA  did  not  occur.  A  change  to  the  production  baseline  without  the 
formal  engineering  change  process  could  impact  product  reliability. 

Purchasing  (7.4) 

Purchasing  Process  (7.4.1).  The  supplier  management  plan  for  the  Development 
and  Sustainment  Contract  had  not  been  approved.  The  Supplier  Management  Plan 
identifies  the  criticality  of  items  used  on  the  EKV  as  well  as  the  procurement  information 
for  long  lead  time  items.  This  is  the  first  contract  with  full  implementation  of  the 
MDA  Mission  Assurance  Provisions.  The  MDA  Mission  Assurance  Provisions  would 
ensure  design,  manufacturing,  test,  and  quality  requirements  were  flowed  down 
to  all  mission-critical  suppliers.  Without  the  supplier  management  plan  identifying 
mission-critical  suppliers,  there  is  no  assurance  that  essential  requirements  were 
flowed  down. 

Purchasing  Information  (7.4.2).  Raytheon  did  not  always  provide  mission-critical 
item  information  to  its  suppliers.  For  example,  Raytheon  did  not  identify  all  critical 
items  within  its  product  data  management  system;  and  critical  item  information 
was  not  always  included  in  purchasing  orders  to  subtier  suppliers.  Mission-critical 
hardware  and  software  items  require  more  stringent  controls  to  ensure  product 
reliability  and  safety.  Designation  of  an  assembly  as  a  mission-critical  item  invokes 
additional  requirements,  such  as  configuration  audits,  first  article  test  and  inspection. 
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piece  part  traceability,  and  further  process  controls  and  monitoring.  If  the  supply 
chain  is  unaware  that  components  or  assemblies  are  critical  items,  then  appropriate 
requirements  will  not  be  imposed. 

Validation  of  Purchased  Product  (7.4.3).  Ra3d:heon’s  Hardware  Acceptance  Review 
Checklists  are  used  to  identify  the  status  of  any  open  or  unresolved  actions  against 
hardware  and  software.  We  found  several  reports  had  not  been  properly  closed. 
For  example,  the  Hardware  Acceptance  Review  Checklist  for  an  inertial  measurement 
unit,  shipped  from  Northrop  Grumman  to  Raytheon,  did  not  identify  whether  that 
unit  had  been  accepted  or  rejected.  The  quality  note  on  the  purchase  order  required 
the  Raytheon  representative  at  Northrop  Grumman  to  accept  or  reject  the  Hardware 
Acceptance  Review  Checklist.  Instead,  this  unit  was  shipped  using  a  "ship  at  risk" 
contract  letter  before  completion  of  the  Hardware  Acceptance  Review  Checklist.  This 
poses  a  risk  of  incorporating  unresolved  quality  issues  into  the  next  higher  assembly. 

Production  and  Service  Provisions  [7.5] 

Control  of  Production  and  Service  Provision  (7.5.1).  During  the  inspection  we 
found  an  instance  where  an  operation  was  stopped  because  the  parts  kit,  which 
went  through  the  kit  audit  process,  contained  a  screw  without  threads.  Further  review 
of  the  kitting  process  identified  that  required  verifications  of  kitted  hardware  were 
not  being  thoroughly  conducted  and  completed.  It  was  also  noted  that  Raytheon 
instructions  did  not  sufficiently  describe  the  kit  audit  process.  A  breakdown  in 
rigor  and  adequately  documented  processes  will  result  in  producing  nonconforming 
parts  and  material  being  issued  to  the  production  floor  for  buildup  of  EKVs. 

Preservation  of  Product  (7.5.5).  Ra3d:heon’s  shipping  requirements  and  processes 
for  the  Inertial  Measurement  Unit  (IMU)  were  not  sufficient.  The  IMU  specification 
did  not  include  a  shipping  requirement  for  prohibition  against  X-ray  inspection  during 
commercial  flight  transportation.  Raytheon  did  not  have  a  documented  process 
to  prevent  X-ray  overexposure,  and  since  the  IMU  may  be  damaged  by  prolonged 
or  numerous  X-ray  exposures,  special  transportation  instructions  are  necessary  for 
product  preservation. 

Monitoring  and  Measuring  Equipment  (8.2) 

Raytheon  was  not  meeting  software  defect  containment  requirements.  Software 
defect  containment  ensures  that  software  issues  are  detected  and  corrected  before 
going  to  the  next  phase  of  software  development.  The  defect  containment  per-phase 
goal  was  set  between  90  and  95  percent;  however,  for  March  2013,  the  actual  defect 
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containment  during  each  phase  was  as  follows:  requirements  phase  42  percent; 
design  phase  39  percent;  the  code/unit  test  phase  34  percent;  and  integration  test 
phase  38  percent.  Ra3d:heon  also  did  not  have  any  information  on  corrective  actions 
taken  to  correct  the  low  defect  containment.  The  cost  and  time  to  correct  defects 
increases  exponentially  as  time  elapses  from  the  time  defects  are  introduced.  Lack 
of  immediate  corrective  action  increases  the  risk  of  defects  remaining  in  the  software 
as  well  as  potential  schedule  slips. 

Control  of  Nonconforming  Product  (8.3) 

Boeing  and  Ra3d:heon  did  not  obtain  MDA  approval  for  some  waivers  and 
deviations  that  were  classified  as  "critical"  or  "major."  Three  deviations  were  noted 
for  Payload  52  that  included  modifications  to  the  Isolated  IMU,  adapter  plate,  and 
accelerometers,  as  well  as  a  fourth  deviation  for  the  use  of  silver  plated  copper  wire. 
In  addition,  Boeing  had  not  approved  the  technical  adequacy  of  the  engineering 
disposition,  but  had  concurred  in  classification  only.  Without  MDA’s  approval 
for  major  deviations,  Ra3d:heon  may  be  delivering  hardware  that  does  not  meet 
requirements.  Also,  Quality  Notifications  were  not  always  documented  for 
nonconformances  generated  on  Manufacturing  Lien  Authorizations  (MLAs).  The  MLA 
is  an  authorization  to  allow  the  movement  of  an  incomplete  item  into  production. 
The  Raytheon  MLA  instruction  states,  "the  MLA  process  is  NEVER  used  to  process 
hardware  with  discrepancies,  unless  used  in  conjunction  with  a  Delayed  Disposition." 
We  noted  several  examples  where  nonconformances  were  documented  on  MLAs 
but  did  not  have  associated  Quality  Notifications.  We  also  noted  several  examples 
of  Quality  Notifications  that  were  written  but  were  not  identified  within  the  MLA. 
Without  properly  documenting  hardware  issues  on  Quality  Notifications,  these 
issues  may  not  receive  adequate  engineering  review  and  disposition.  As  a  result, 
nonconformances  may  circumvent  the  engineering  review  board  and  material 
review  board  processes.  Approval  of  deviations  ensures  that  MDA  is  aware  of  any 
potential  impacts  to  the  system  that  could  result  from  incorporation  of  hardware 
or  software  that  is  not  produced  to  specifications. 
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Boeing 

Our  assessment  of  the  EKV  at  Raytheon  also  included  review  of  Boeing’s  management 
of  Raytheon  as  its  supplier. 

Customer  Focus  (5.2). 

Boeing  had  approved  modifications  to  the  Ra3d:heon  Mission  Assurance  Provisions 
Requirements  Applicability  Matrix  even  though  there  was  inadequate  rationale  to 
justify  the  modifications,  particularly  for  new  and  modified  hardware  and  software. 
Inadequate  justifications  existed  for  the  following  requirements:  Sneak  Circuit 
Analysis,  Qualification/Re-Qualification  Test  Program,  Workmanship  Standards, 
Product  Identification  and  Handling,  System  Safety  Program  Plan,  Design  and 
Development  of  Computer  Systems,  and  Interface  Design  Requirements.  The  purpose 
of  the  Requirements  Applicability  Matrix  was  for  Raytheon  to  show  Boeing  how 
they  met  the  intent  of  the  MDA  Mission  Assurance  Provisions.  Modifications  and,  or 
tailoring  of  the  requirements  is  allowed  provided  that  a  sound  engineering  rationale 
is  provided.  Changes  to  mission-critical  requirements  without  adequate  rationale 
may  result  in  a  product  that  does  not  meet  customer  requirements  and  increased 
program  performance,  cost,  and  schedule  risks. 

Planning  of  Product  Realization  [7.1] 

Configuration  Management  (7.1.3).  Boeing  was  allowing  design  changes  without 
using  the  appropriate  engineering  documentation  necessary  to  control  the  configuration 
changes  of  the  EKV  baseline.  Boeing  deferred  requirements  indefinitely,  exempting 
requirements  through  a  contract  letter  without  the  rigor  of  established  change 
management  processes,  and  incorrectly  recategorized  waiver  requests.  These  conditions 
can  also  result  in  Boeing  delivering  products  that  do  not  meet  requirements. 

Design  and  Development  (7.3) 

Boeing’s  critical  supplier  list  did  not  include  all  the  mission-critical  suppliers  that 
Raytheon  identified  as  "critical  suppliers."  These  included  suppliers  for  harnesses, 
circuit  card  assemblies,  aft  covers,  and  beamsplitters.  Suppliers  of  critical  items 
require  additional  controls  and  oversight  to  include  flowdown  of  MDA  assurance 
provisions,  prework  authorization  reviews,  postaward  reviews,  and  product  and 
process  verification  assessments.  If  suppliers  of  mission-critical  items  are  not  included 
on  the  critical  supplier  list,  the  additional  controls  and  oversight  required  to  ensure 
product  integrity  of  the  parts  and  overall  system  will  not  be  implemented  and  this 
may  lead  to  quality  escapes  and  performance  degradation. 
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Corrective  Actions  Taken  to  Date 

According  to  the  GMD  Program  Office,  they  have  prepared  corrective  action  plans  for 
the  nonconformances  identified.  MDA,  Boeing,  and  Raytheon  are  working  together 
to  ensure  that  all  of  the  findings  and  recommendations  are  understood,  a  root  cause 
analysis  is  performed,  and  corrective  action  plans  are  initiated.  Table  3  shows  the 
status  of  corrective  action  plans  for  each  site  as  of  July  24,  2014. 


Table  3.  Contractor  Nonconformance  Closeout  Status 


Responsible  Contractor 

Nonconformances 

In  Work 

Complete 

Boeing 

7 

0 

7 

Raytheon 

40 

4 

36 

GMD  Program  Office 

1 

0 

1 

Total 

48 

4 

44 

Source:  GMD  Program  Office,  July  24,  2014 
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Finding  A 

Software  Management  Practices 

We  found  a  significant  number  of  AS9100C  nonconformances  related  to  software 
development  and  testing.  Boeing  and  Raytheon  were  not  ensuring  that  software 
development  processes  and  testing  were  sufficient.  This  lack  of  process  discipline  and 
controls  could  result  in  system  reliability  risks. 


The  significant  number  of  software  nonconformances  indicated  a  lack  of  process 
discipline: 


•  Software  Development  -  Raytheon  did  not  always  adhere  to  all  software 
coding  conventions  and  standards. 

•  Corrective  Action  and  Verification  -  Raytheon  was  not  meeting  software 
defect  containment  requirements  and  did  not  take  corrective  action. 

•  Software  Testing  -  For  one  EKV  software  release,  not  all  required  safety 
critical  software  test  cases  were  conducted  for  the  CE-1  formal  delta 
qualification  testing. 

Recommendation  A  -  Management  Comments  and 
Our  Response 

We  recommend  that  the  Director,  Missile  Defense  Agency  ensure  software 
development  processes  are  fully  documented,  implemented,  and  enforced 
throughout  the  Exoatmospheric  Kill  Vehicle  supply  chain. 

Director,  Missile  Defense  Agency 

Director,  Missile  Defense  Agency  agreed  and  projects  that  by  the  first  quarter  of 
fiscal  year  2015,  MDA  will  correct  and  effectively  implement  actions  to  address 
weakness  within  software  development,  verification,  and  validation  process;  the 
training  program;  and  the  testing  of  critical  safety  items. 

Our  Response 

The  Director’s  comments  are  responsive.  We  request  that  MDA  notify  the  OIG  when  the 
actions  are  complete.  No  further  comments  are  required. 
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Finding  B 

Supply  Chain  Management 

Boeing  and  Raytheon  supplier  management  processes  did  not  meet  AS9100C  quality 
standards.  They  did  not  ensure  all  quality  assurance  and  technical  requirements 
for  mission-critical  assemblies  flowed  down  to  the  supply  chain  and  were  verified. 
This  could  result  in  suppliers  delivering  nonconforming  hardware  or  software,  thus 
reducing  system  reliability. 


We  noted  a  significant  number  of  nonconformances  in  the  following  areas. 

•  Supply  Chain  Management  Planning  -  Raytheon  did  not  have  an  approved 
Supplier  Management  Plan  for  the  Development  and  Sustainment  Contract. 

•  Design  and  Development  Requirements  -  Boeing  and  Raytheon  did  not 
always  flow  down  all  necessary  quality,  safety,  and  mission  assurance 
requirements  for  critical  item  suppliers. 

•  Verification  of  Purchased  Product  -  Raytheon  did  not  complete  the 
Hardware  Acceptance  Review  Checklist  process  before  allowing  hardware 
to  be  shipped. 

Recommendation  B.l-  Management  Comments  and  Our 
Response 

We  recommend  that  the  Director,  Missile  Defense  Agency  ensure  all  suppliers 
of  critical  items  are  identified  as  critical  suppliers,  receive  the  necessary 
contractual  requirements,  and  requirements  are  verified  throughout  the 
supply  chain. 

Director,  Missile  Defense  Agency 

The  Director,  Missile  Defense  Agency  agreed  and  stated  that  MDA  is  identifying 
suppliers  of  critical  items  to  ensure  the  flow  down  of  quality  and  mission  assurance 
requirement  to  those  suppliers. 

Our  Response 

The  Director’s  comments  are  responsive.  We  request  that  MDA  notify  the  OIG  when  the 
actions  are  complete.  No  further  comments  are  required. 
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Recommendation  B.2  -  Management  Comments  and 
Our  Response 

We  recommend  that  the  Director,  Missile  Defense  Agency  ensure  fielded  hardware 
affected  by  an  insufficient  Hardware  Acceptance  Review  Checklist  process  is 
assessed  for  risk. 

Director,  Missile  Defense  Agency 

The  Director,  Missile  Defense  Agency  agreed  and  stated  that  MDA  is  conducting  a  risk 
assessment  on  fielded  hardware  affected  by  insufficient  requirements  and  the  results 
of  the  assessment  will  be  incorporate  into  the  MDA  continuous  improvement  plan. 

Our  Response 

The  Director’s  comments  are  responsive.  We  request  that  MDA  notify  the  OIG  when 
the  actions  are  complete.  No  further  comments  are  required. 
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Finding  C 

Configuration  Management 

Boeing  and  Raytheon  management  and  execution  of  design  variances  and  engineering 
changes  did  not  meet  AS9100C  requirements.  Boeing  and  Raytheon  were  not  adhering 
to  configuration  management  processes,  specifically  with  respect  to  management  of 
change  processes  for  design  requirements.  The  lack  of  adherence  to  these  disciplines 
can  result  in  the  fielding  of  an  unapproved  configuration. 


We  noted  a  significant  number  of  nonconformances  in  the  following  areas. 

•  Design  Variance  and  Engineering  Changes  -  Raytheon  was  using  waiver 
and  deviation  process  instead  of  the  engineering  change  process  to  modify 
the  production  baselines.  Additionally,  Boeing  used  contract  letters  to 
implement  changes  to  the  design  and  performance  specification  instead  of 
the  appropriate  change  management  process.  Not  following  the  required 
process  could  increase  the  risk  of  ineffective  change  control. 

•  Classification  of  Variance  -  Boeing  and  Raytheon  were  incorrectly  classifying 
waivers  and  deviations,  resulting  in  changes  to  the  baseline  without 
appropriate  approval. 

Recommendation  C  -  Management  Comments  and 
Our  Response 

We  recommend  that  the  Director,  Missile  Defense  Agency  ensure  design  and 
configuration  changes  do  not  circumvent  the  Missile  Defense  Agency  Assurance 
Provisions  for  configuration  management. 

Director,  Missile  Defense  Agency 

The  Director,  Missile  Defense  Agency,  agreed  and  stated  that  MDA  is  updating  the 
Configuration  Management  process  to  incorporate  omitted  engineering  change 
processes,  properly  characterize  major  and  minor  variances  within  the  process,  and 
actively  manage  the  waiver/deviation  process. 

Our  Response 

The  Director’s  comments  are  responsive.  We  request  that  MDA  notify  the  OIG  when  the 
actions  are  complete.  No  further  comments  are  required. 
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Finding  D 

Overall  Findings  and  Recommendations 

We  identified  a  total  of  48  nonconformances  that  were  violations  of  the  AS9100C 
standard.  Based  on  these  nonconformances,  we  determined  that  MDA,  Boeing, 
and  Raytheon  were  not  ensuring  that  the  all  quality  management  systems  were  in 
compliance  with  the  AS9100C  standard.  These  nonconformances  could  result 
in  the  production  of  nonconforming  hardware  and  software  which  could  effect 
mission  success. 


The  nonconformances  represent  very  specific  quality  management  system  issues 
requiring  resolution.  It  appears,  from  the  data  provided,  that  the  GMD  Program 
Office  has  a  process  to  identify  root  causes  and  put  corrective  action  plans  in  place, 
which  we  acknowledge  and  appreciate.  It  is  our  professional  judgment  that  the 
following  three  primary  findings  represent  the  most  problematic  areas  requiring 
special  attention:  Software  Management  Practices,  Supply  Chain  Management,  and 
Configuration  Management.  Although  we  found  the  majority  of  quality  management 
systems  in  compliance  with  the  AS9100C  standard,  the  importance  of  these 
three  quality  assurance  categories  cannot  be  overstated  in  terms  of  improving 
reliability  growth. 

Recommendation  D  -  Management  Comments  and 
Our  Response 

We  recommend  that  the  Director,  Missile  Defense  Agency  conduct  an  effective 
root  cause  analysis  and  implement  corrective  actions  for  all  48  nonconformances 
including  assessing  the  risk  to  fielded  hardware. 

Director,  Missile  Defense  Agency 

The  Director,  Missile  Defense  Agency,  agreed  and  stated  that  MDA  is  conducting  a 
root  cause  analysis  and  developing  corrective  actions  for  deficiencies  identified  during 
the  onsite  assessment  and  will  assess  the  risk  to  fielded  hardware. 

Our  Response 

The  Director’s  comments  are  responsive.  We  request  that  MDA  notify  the  OIG  when  the 
actions  are  complete.  No  further  comments  are  required. 
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Appendix  A 

Scope  and  Methodology 

We  conducted  this  assessment  from  September  2013,  through  August  2014,  in 
accordance  with  the  Council  of  the  Inspectors  General  on  Integrity  and  Efficiency, 
"Quality  Standards  for  Inspection  and  Evaluation."  Those  standards  require  that 
we  plan  and  perform  the  assessment  to  obtain  sufficient,  appropriate  evidence  to 
provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  assessment 
objectives.  We  believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  assessment  objectives. 

To  evaluate  the  management  of  the  EKV  quality  assurance  program,  we  performed  a 
quality  assurance  assessment  of  the  MDA  GMD  Program  Office  in  Huntsville,  Alabama, 
and  Ra3d:heon  Missile  Systems  in  Tucson,  Arizona.  To  review  the  quality  management 
system,  our  assessments  focused  on  the  following: 

•  Applicable  Statutory/regulatory  requirements 

•  Contractual  quality  management  system  (AS9100C] 

•  Internal  quality  assurance  processes  and  procedures 

We  reviewed  program  office  documentation,  including  EKV  configuration  differences, 
field  asset  reliability  data,  and  program  office  risk  management  information.  At 
Raytheon,  we  reviewed  contractual  requirements,  manufacturing  and  quality 
management  system  documentation,  failure  reporting  data,  and  waivers  and  deviations. 
Because  both  CE-I  and  CE-II  configurations  are  in  use,  we  compared  select  builds  to 
note  changes  in  the  quality  management  system  processes  and  procedures. 

We  issued  findings  and  recommendations  commensurate  with  our  assessment  of 
the  quality  assurance  and  reliability  of  the  GMD,  EKV,  which  was  procured  through 
contract  effort.  We  did  not  assess  or  comment  upon  the  consequences  of  our  findings 
as  they  relate  to  the  contractual  rights  and  obligations  of  the  parties  involved  nor 
any  resulting  contracting  actions  that  may  be  available  to  MDA. 
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DoD  OIG  Assessment  Criteria 

^5  9100C,  Quality  Management  Systems  -  Requirements  for  Aviation, 

Space  and  Defense  Organizations 

AS9100C  was  contractually  imposed  on  Boeing  and  flowed  down  to  Raytheon  for 
the  DSC,  awarded  in  June  2012,  by  way  of  the  MDA  Assurance  Provisions  (MAP]. 
The  MAP  establishes  requirements,  standards,  and  policies  for  quality,  safety,  and 
mission  assurance.  Therefore,  we  used  AS9100C  standard  as  our  assessment  criteria. 
The  AS9100C  standard  breaks  down  quality  assurance  requirements  into  five 
major  clauses: 

•  Quality  Management  System; 

•  Management  Responsibility; 

•  Resource  Management; 

•  Product  Realization,  and 

•  Measurement,  Analysis,  and  Improvement 

The  Quality  Management  System,  Management  Responsibility,  and  Resource 
Management  clauses  of  AS9100C  require  the  organization  to  have  a  quality  assurance 
management  organization  that  has  all  the  resources  and  authority  to  affect  the 
end-item  quality  of  the  product.  In  addition,  these  clauses  require  the  organization 
to  have  a  quality  assurance  manual  with  strict  controls  over  all  documentation,  data, 
and  procedures  that  affect  the  quality  of  the  product.  Product  Realization  covers  the 
activities  and  processes  necessary  to  bring  a  product  into  existence.  Measurement, 
Analysis,  and  Improvement  requires  the  organization  to  ensure  the  product 
continuously  improves.  The  clause  includes  customer  satisfaction,  internal  audit, 
monitoring  and  measuring  processes  and  product,  and  control  of  nonconforming 
products  to  ensure  continual  improvement. 

Product  realization  is  broken  down  further  in  AS9100C  as  follows: 

•  Planning  of  Product  Realization, 

•  Customer-Related  Processes, 

•  Design  and  Development, 
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•  Purchasing, 

•  Production  and  Service  Provision,  and 

•  Control  of  Monitoring  and  Measuring  Equipment 

Planning  of  Product  Realization  requires  the  organization  to  develop  processes 
needed  for  design  and  development  of  product  and  includes  elements  such  as 
procedures,  quality  assurance  records,  resource  requirements,  safety  and  reliability 
programs,  and  inspection  and  test.  Customer-Related  Process  requires  the 
organization  to  determine  customer  requirements  both  specified  and  derived.  These 
requirements  include  technical,  statutory,  and  regulatory  requirements.  Design  and 
Development  includes  requirements  that  cover  planning,  inputs,  outputs,  review, 
verification,  validation,  and  control  of  changes  as  related  to  design  and  development. 
Purchasing  requires  the  organization  to  ensure  that  the  purchased  product  conforms 
to  specified  purchase  requirements  and  that  all  products  purchased  from  suppliers 
are  verified  against  purchase  agreement  requirements.  The  Production  and  Service 
Provision  requires  the  organization  to  ensure  that  production  is  accomplished 
under  controlled  conditions  using  drawings  and  specifications,  work  instructions, 
production  tools  and  software  programs,  monitoring  and  measuring  equipment, 
and  evidence  that  all  production  and  inspection/verification  operations  have  been 
completed  as  planned.  Control  of  Monitoring  and  Measuring  Equipment  requires 
the  organization  to  ensure  that  devices  used  for  determining  product  compliance 
with  performance  characteristics  are  properly  maintained  to  provide  assurance  of 
credible  measurements. 

Use  of  Technical  Assistance 

Quality  assurance  engineers  and  quality  assurance  specialists  with  a  background  in 
defense  assisted  in  the  assessment.  We  established  teams  of  subject  matter  experts 
who  assessed  to  the  AS9100C  Quality  Management  System  standard.  The  subject 
matter  expert  teams  consisted  of  17  quality  assurance  engineers  who  have  received 
AS9100C  certification  training,  and  have  an  average  of  17  years  of  quality  assurance 
audit  experience. 
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Appendix  B 

Prior  Coverage 

During  the  last  5  years,  the  Government  Accountability  Office  (GAO)  issued  eight 
reports  discussing  the  Ground-Based  Midcourse  Defense,  Exoatmospheric  Kill  Vehicle. 
Unrestricted  GAO  reports  can  be  accessed  over  the  Internet  at  http://www.gao.gov. 

GAO 

Report  No.  GAO-13-294SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  28,  2013 

Report  No.  GAO-12-486,  "Missile  Defense:  Opportunity  Exists  to  Strengthen  Acquisitions 
by  Reducing  Concurrency,”  April  20,  2012 

Report  No.  GAO-12-400SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  29,  2012 

Report  No.  GAO-11-555T,  "Missile  Defense:  Actions  Needed  to  Improve  Transparency 
and  Accountability,"  April  13,  2011 

Report  No.  GAO-11-233SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  29,  2011 

Report  No.  GAO-11-372,  "Missile  Defense:  Actions  Needed  to  Improve  Transparency 
and  Accountability,"  March  24,  2011 

Report  No.  GAO-10-311,  "Defense  Acquisitions:  Missile  Defense  Transition  Provides 
Opportunity  to  Strengthen  Acquisition  Approach,"  February  25,  2010 

Report  No.  GAO-09-403T,  "Defense  Acquisitions:  Charting  a  Course  for  Improved 
Missile  Defense  Testing,”  February  25,  2009 
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Missile  Defense  Agency 


DoD  IG  Project  DTOTAD-0005 
Exoatmospheric  Kill  Vehicle  Quality  Assurance  Assessment 
DRAFT  REPORT  -  Part  A 
July  2,  2014 

Missile  Defense  Agency  Response  to  DoD  IG  Recommendations 

RECOMMENDATIONS 

Recommendation  A:  Ensure  software  development  proeesses  are  fully  doeumented, 
implemented,  and  enforeed  throughout  the  EKV  supply  ehain. 


MPA  Resyonse  to  Recommendation  A:  Concur. 

Those  portions  of  the  Software  (SW)  development  processes  that  may  lack  verification  and 
validation  process,  suitable  training  program,  proper  SW  Coding,  and  critical  safety  testing  will  be 
corrected  and  effectively  implemented.  MDA  corrective  actions  will  be  monitored  and  reviewed  as  a 
part  of  the  Contractor  Performance  Assessment  Report  (CPAR)  and  the  Quality  Assurance 
Surveillance  Plan  (QASP)  to  ensure  conformance  and  commitment  to  the  corrective  actions.  The 
MDA  corrective  actions  findings  regarding  the  software  development  processes  are  projected  to  be 
completed  by  1 QFY 1 5 . 

Recommendation  Bl:  Ensure  all  suppliers  of  eritieal  items  are  identified  as  eritieal  suppliers 
reeeive  the  neeessary  eontraetual  requirements,  and  requirements  are  verified  throughout  the 
supply  ehain. 

MDA  Response  to  Recommendation  Bl:  Concur. 

MDA  will  correct  and  effectively  implement:  Quality  and  mission  assurance  requirement  flow-down 
for  critical  suppliers  that  may  lack  all  critical  items  identification  and  critical  information  contained 
in  purchasing  orders  to  sub-tier  suppliers;  supplier  management  conformed  to  approved  supplier 
management  plan  for  the  Development  and  Sustainment  Contract;  and  verification  of  purchased 
items.  MDA  is  ensuring  suppliers  of  critical  items  are  identified  as  critical  suppliers  and  receive  the 
necessary  contractual  requirements  and  the  requirements  are  verified  throughout  the  supply  chain. 
The  corrective  actions  will  be  monitored  and  reviewed  as  a  part  of  the  CPAR  and  the  QASP  to 
ensure  conformance  and  commitment  to  the  corrective  actions. 

Recommendation  B2:  Ensure  Fielded  hardware  affeeted  by  insuffieient  requirements  is 
assessed  for  risk. 

MDA  Response  to  Recommendation  B2:  Concur. 

MDA  is  ensuring  fielded  hardware  affected  by  insufficient  requirements  is  assessed  for  risks.  The 
corrective  actions  associated  with  this  recommendation  will  be  monitored  and  reviewed  as  a  part  of 
the  CPAR  and  the  QASP  to  ensure  conformance  and  commitment  to  the  corrective  actions.  Results 
of  the  risk  management  processes  shall  be  used  for  continuous  process  improvement. 
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Missile  Defense  Agency  (cont'd) 


Recommendation  C:  Ensure  design  and  eonfiguration  ehanges  do  not  eireumvent  the  Missile 
Defense  Ageney  Assuranee  Provisions  for  eonfiguration  management. 

MPA  Response  to  Recommendation  C:  Concur 

The  Configuration  Management  process  will  incorporate  any  omitted  engineering  change  processes, 
properly  characterize  major  and  minor  variances,  and  actively  manage  the  waiver/deviation  process. 
MDA  is  ensuring  that  the  design  and  configuration  changes  meet  the  Missile  Defense  Agency 
Assurance  Provisions  (MAP)  for  configuration  management.  The  corrective  actions  associated  with 
this  recommendation  will  be  monitored  and  reviewed  as  a  part  of  the  CPAR  and  the  QASP  to  ensure 
conformance  and  commitment  to  the  corrective  actions. 

Recommendation  D:  Conduct  effective  root  eause  and  eorreetive  aetions  for  all  defieieneies 
ineluding  assessing  the  risk  to  fielded  hardware. 

MDA  Response  to  Recommendation  D:  Concur 

MDA  is  conducting  root  cause  analysis  and  developing  corrective  actions  for  deficiencies  including 
assessing  the  risk  to  fielded  hardware.  The  corrective  actions  associated  with  this  recommendation 
will  be  monitored  and  reviewed  as  a  part  of  the  CPAR  and  the  QASP  to  ensure  conformance  and 
commitment  to  the  corrective  actions.  The  projected  completion  date  is  1QFY15. 
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Acronyms  and  Abbreviations 


AS  Aerospace  Standard 
BMDS  Ballistic  Missile  Defense  Systenn 
CE  Capability  Enhancennent 
DoDD  Departnnent  of  Defense  Directive 
DoDI  Departnnent  of  Defense  Instruction 
DSC  Developnnent  and  Sustainnnent  Contract 
EKV  Exoatnnospheric  Kill  Vehicle 
FMECA  Failure,  Modes,  Effects,  and  Criticality  Analysis 
FTG  Flight  Test  Ground-Based  Interceptor 
GBI  Ground-Based  Interceptor 
GMD  Ground-Based  Midcourse  Defense 
IMU  Inertial  Measurennent  Unit 
MAP  MDA  Assurance  Provisions 
MDA  Missile  Defense  Agency 
MLA  Manufacturing  Lien  Authorization 

NSPD  National  Security  Presidential  Directive 
OIG  Office  of  Inspector  General 
QMS  Quality  Managennent  Systenns 
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Whistleblower  Protection 

U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions 
on  retaliation,  and  rights  and  remedies  against  retaliation  for 
protected  disclosures.  The  designated  ombudsman  is  the  DoD  Hotline 
Director.  For  more  information  on  your  rights  and  remedies  against 
retaliation,  visit  www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

congressional@dodig.mil;  703.604.8324 

Media  Contact 

public.affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report@listserve.com 

Twitter 

twitter.com/DoD_IG 

DoD  Hotline 

dodig.mil/hotline 
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